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3.4 REPORT OF THE SUBPANEL ON INTER-IM* MATCHING 


The inter-image matching panel met for two sessions to define research needs 
in this subject area for earth resource observation systems. The panel con- 
sisted of: 


Mr. H. K. Ramapriyan 
NASA/GSFC 

Mr. R. Bryan Erb 
JSC 

Ms. June Thormodsgard 
USGS 

Mr. Jerry M. Cantril 
Computer Sciences-Technicolor 

Mr. Frederick W. McCaleb 
NASA/GSFC 


Mr. Donald Brandshaft 
Santa Barbara Research Center 

Mr. Paul F. Maynard 
Earth Satellite Corporation 

Mr. Terry Silverberg 
U of MD 

Nr. Richard Sigman 
Department of Agricul ture 

Professor Laveen Kanal 
U of MD 


Mr. George Austin 
Lockheed 


The survey paper prepared by R. Wolfe and P.. Juday provided a good overview of 
the state of the art and requirements. The panel concentrated upon developing 
the design of and rationale for research efforts in this area. 

3.4.1 Recommended Research 

Six research areas were identified by the panel and are described below: 

I. Correlation Considerations 

The issues to be addressed under this topic are those relating to matching 
small sections of images (or information derived from images) to other images 
(or map-derived information) . We recommand considering the following topics: 
a) types of correlation algorithms, b) preprocessing techniques, c) limits on 
subpixed accuracy, d) relief and atmospheric refraction effects, 
e) evaluation of success rate, f) avaluation of accuracy. 

We are aware of various matching and preprocessing techniques as indicated in 
the survey paper. However, a study is still needed to compare these from the 
points of view of interactions among them and their effects on accuracy of 
matches. A study should be made of the theoretical limits on the subpixed ac- 
curacy that can be achieved given that the images are sampled data. The va- 
lidity of techniques used for estimating "subpixel offsets" (e.g., surface 
fitting) for matching maps should be examined. The effects of relief and at- 
mospheric refraction on the image-matching process should be quantified, es- 
pecially in relation to higher resolution sensors. Techniques should be de- 
veloped for estimating probabilities of image matching success, false-alarm 
and miss rates. 


33 



The correlation techniques to be considered should include **nonraster" data 
types* Extraction of image features to match with ^digital map” representa- 
tions should be considered due to its potential for a common data base for 
multiple sensors. Different data types may require different algorithms. 
Both sensor data correlation and the recuired preprocessing should be studied. 
A tradeoff study should be made to find the optimum combination of preprocess- 
ing correlation and surface fitting for offset estimation in relation to vari- 
ous sensor types (resolution* spectral characte’^istics, geometries), ground 
characteristics, and imaging seasons. 

If correlation techniques guaranteeing successful image matching are not 
available and verified, the specifications such as the Thematic Mapper equip- 
ment of .3 (relative) and .5 (absolute) accuracy cannot be met. It has been 
shown that hard specifications such as these imply high ground system costs. 
Therefore it is necessary to establish realizability of such accuracies so 
that we either not be as stringent with the specifications and let users ”live 
with” the errors, or come up with techniques for achieving and verifying the 
accuracies. Multisensor image matching and image to nonimage data matching 
techniques hsve the potential of saving the effort needed in building multiple 
control point libraries. 

Items a through d listed below constitute a coherent group. The prioritiza- 
tion should be in terms of theoretical work and he experiments to be per- 
formed on various sensor types, data types, etc The sequence of events 

should be as follows: a) create a common software brary to be shared among 

researchers; b) perform theoretical studies of j pixed accuracies, relief 
distortions and atmospheric refraction effects; c) identify sensor types and 
experimental data sets (synthetic and real) to be u:ed; d) conduct tests of 
technicians and trade off studies. 

II. Study to determine interaction of inter-image matching procedures and se- 
lection of GCPs for a given sensor . 

The types of GCPs which should be selected for future libraries will most 
likely depend not only on the characteristics of the GCP, but also on the 
matching procedures in which the GCPs will be used. For remote sensing data 
from a given sensor , an experimental design with the following analysis ele- 
ments is proposed: a) preprocessing procedure analysis, b) inter-image match- 

ing factors analysis (correlation measure, correlation peak determination, 
match reject/ accept criteria), c) GCP factors analysis (type of GCP, season 
and scene characteristics of base chip, season and scene characteristics of 
test chip) . 

Within each factor combination, chip pairs with known image-to- image matches 
(determined manually) will be subjected to the processing factors. Measured 
response data for the experimental design will be matches and summary statis- 
tics (average, dispersion, percentiles) of the difference between true offset 
and automatically determined offset for nonrejected matches, and measures of 
the temporal stability of matching performance. The benefits of the above de- 
scribed study are that the study conclusions will provide guidance in selec- 
tion of GCPs for future libraries. The penalties for not performing the study 
is that future GCP libraries will be built but will have low success rate in 
automatically registering data. 



III. Global multisensor control point library feasibility study. 

If ground control points are to be used to rectify and/or register future re- 
motely sensed image data, the feasibility of a highly automated, global, mul- 
tisensor control point library should oe determined. Such a system should in- 
corporate improved methods of loading control points. Improved user access 
(Including the capability for users to interactively load their own control 
points from remote terminals), standard control point formats, and a multijen- 
sor data base. 

A global multisensor control point library would reduce the operational costs 
associated with precision rectification and registration of future sensor sys- 
tems, In addition, the capability for users to load special sets of GCPs and 
access the GCP library allows the user to guarantee the rectification and/or 
registration accuracy of specific data sets of interest to him. 

The Initial capital outlay to implement such a system would probably be in ex- 
cess of SiO milli-:)n. In addition, multiseiisor registration may well not be 
easy to achieve. 

IV. Multiple Control Point Considerations 

Elevation of observed control point mismatches to an image-level mismatch 
charaoterizstiou is effected by fitting a distortion function to the GCP off- 
sets. This distortion function could arise from (1) Inadequacy 5n £ priori 
definition of the geometrical model (e.g., not Including scan anomelies, ter- 
rain effects, atmospheric differential refraction, etc.), (2) from an incom- 
plete characterization of its form (e.g., role is modeled as third-order in 
time but the dota is first order), or (3) the case that the a priori model is 
properly characterized but with imprecise parameters (e.g., polynomlnal coef- 
ficients are refined with GCP data). Characterization of the model dictates 
the optimal sizing and placement of control points. The possibility of unde- 
tected GCP outliers (i-e,, erroneous matches) dictates the need for estimation 
techniqu'^s beyond conventional least squares. 

The proposed research would investigate the effect of number and distribution 
of control points required as an effect of image (sensor) characteristics ade- 
quacy of fit to an a priori model or selection of degree or model for warping 
function in an adaptive senso. The adaptive capability will reduce the number 
of correisbions required for mapping well ordered data sets and consequently 
“■he processing time or equipment complexity while extending tne capability to 
Handle diverse data sets at limited througnput rates. 

The following specific areas are recommended: a) Investigate spatial distor- 
tion filtering: derive new techniques or expand existing techniques, e.g. 
Kahlman filtering; b) Test polynomial rubber sheeting in terms of compromise 
between order selection and reducing i3MS error; c,’ investigate application of 
robust estimation techniques to rubber-sheeting and to fitting a priori forms; 
d) Investigate adaptive placement of control points, e.g., to effect minimum 
RMS fit errors; e) Investigate magnitude of high-frequency geomet»'lcal dis- 
tortions V atmospheric distortion, sensor anomalies) and control point sizing; 
f) Investigate methods to overcome geometrical distortions with a GCP patch. 
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Candidate methods include iterative registration (correlate, remap, recorrel- 
ate, remap, etc.) and application of the affine/Fourier technique of Emmert 
and McGillam to determining the offset in an affinely distorted GCP. 

V. Centralized System Study 

A trade-off study needs to be undertaken that considers the optimal hardware 
configuration for image production of data derived from different sensor types 
with emphasis on software/hardware tradeoff for balanced computation and I/O. 
Of particular interest is the read to determine feasibility of developing a 
centralized system to perform registration and rectification for various sen- 
sors (i.e., SAR, MLA, MSS, etc.). Development of a centralized system could 
be the roost economical and efficient approach to meet future image processing 
requirements. 

VI. User System Studies 

By using a combination of systems analysts, a survey of user experience, and 
experimental tests, it should be possible to determine the speedup factors a 
user can expect by using commercially available special purpose hardware ouch 
as array processors and bulk core memory. It should also be possible to de- 
termine whether it is economical to speed up commonly used algorithms by using 
simple custom electronics. This study would assist users in hardware systems 
design. 

3.4.2 Research Priorities 

Priorities fo>" the proposed research are subdivided in Figures 1 and 2 into 
near- or long-term tasks. The elements listed are subsets of the six recom- 
mended research areas. 
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TYPES OF CORRELATION ALGORITHMS 


POOR QUALITY 




{/) 







LU 







►—« 






1 

0 

a: 

z 

en 



z 

z <c 

UJ 

0 

c 



0 

0 a: 


»—4 

0 



z 

»-» zo 

UJ 

1 — 

••— 




»- 0 

z: 

ex 

+-> 



Li. 

c 0 

0 

0 

<0 



0 

a: <c 


t_ 

“D 





Z Q 

JO 

C 



>- 

to 0 


1—4 

m 



I— 

1 — 4 *— * 

•— # 

a 

P 

u 



0 1 - 

l_ ^ 





.J 

UJ C 

<C -J UJ 

Li. 

0 



1 -^ 

OC 'SL 

z: > 

LU 

0 

QJ 


CO 

UJ 

1 — c 

1—4 

CJ 

</> 


»— * 

Q. y- 

to 3 

-J 

ex 

0 ) 


to 

0 L/) 

1— 1 ^0 

LU 


ex 



0 >- 

0 x: tx 

ex 

E 



LU 

1 to 

UJ 1 — 0 


C 

u 


Li. 

Z 

OC ^ 

u. 

OJ 

0 



10 0 

-X 

0 

1 — 

u. 


UJ 

£ UJ 

ex to 





I— 

•-I > 

0 ^ LU 

z 

C 7 ) 



c 

z 0 

L/> X > 

0 

c 



0 

•-4 a: 

z 


c 



»— « 

z: 0 - 

LU LO 

h- 

u 



h- 

2 : 

to ex to 

0 




to 

Qi 

<c < 

LU 



3E 

UJ 

0^ 

1 — 00 CL 

ex 

• 


0^ 

> 


.J 

ex 

CVJ 


UJ 



ZD 

0 



1 — 



X 

CJ» 



0 





rs 


z 












•r~ 


-J 





U. 



01 

C-> 


00 

q: 



0 to 







-C 


<; 0 


00 





u 


ex 


ex 





S- 


c f— 


0 





CO 


:c to 


to 





a) 


(J> »— c 


z 





to 


DT 


UJ 





0 ) 


_J LU 

^ J— 


to 





ex 


ex CD 


H“ 

z 




X 


ts 

>- 

z 

0 




0 


0 

LU 





< 4 - 


UJ <c 

< 

ex 

►— 






ex z 

ex 

LU 

to 



z 

to 


to CD 

ZD 

U. 

LU 

LU 


0 

IZ 



0 

Lu 

ZD 

ex 



0 


>- 0 

CD 

W 4 

ezr 

ZD 

00 

»— 

*r* 


ex z: 

<c 

Q 


h— 

LU 

ex 



h- ZD 



>- 

CD 

ZD 

0 

03 


UJ 0 

Q 

ex 

CD 

UJ 

cr 

h- 

T 3 

to 

X ex 

z z 

0 

<x: 

1 — 


to 

C 

LU 

0 CD 

c 0 

u. 

ex 

►— * 

z 

»— 4 


ZD 

LU 

►— » 


r) 

DZ 

z 

0 


cr 

CD z: 

LU H- 

to 

CD 

CD 

e-> 




0 


- 

0 

ex 

UJ 

Li. 

0 

z 

ex lO 

ex 

cX 

c 

♦— 

LU 

u 

X 

0 < 

ex 

CD 




»— t 

0 ) 

0 

to UJ 

to 

CD 

—1 

UJ 

z 


X 

UJ 

Z t/^ 

to 


LU 

ex 

0 

LU 


1 — 

LU 

to CD 

u. 

X 

<c 


ex 

E 


to z 

UJ UJ 

0 


g 

h- 


X 

CD 

0 

CD ek 


ex 

<C 

u. 

OJ 

Z 

♦ ^ 

CD 

>- 

CO 

ex 


0 

-M 

*—4 

»— H' 

ZD De 

»— 

ZD 

c 

•— » 



to 

. ZD 

10 0 


CO 

ir 

1 — 

z 

X 

to 

ex ^ 

to 

—1 



to 

0 


UJ 

• 0 

LU Z- 

»— 1 

u. 

u_ 

UJ 


a> 

0 

z: to 

H- LU 

CQ 

0 

0 


I— 

z 

0 

LU 

<C tn 

< 




«x 


ex 

ex 

ZD 

»— 

>- 

>• 

to 

ZD 


ex 

0 

— J 1 - 

ex 

0 

0 

ZD 

.J 

• 

UJ 

ZD 

<c ^ 

<c 

ZD 

r> 

esD 


r— 

ex 

H- 

^ i 

0 

h- 

H- 

0 

> 


ex 

to 

c 

to 

to 

ex 

Uj 

0 ) 


Od 


3 

o> 


37 



